Towards comparative evaluations of user-adaptive software systems

Currently there are many user-adaptive systems across different application types including education, business and information systems. However, evaluating user-adaptive systems is still a challenging research issue and a difficult task. This is because of the lack of widely accepted evaluation methods, data and the difficulty on generalizing the application areas. In order to provide comparative and convincing arguments that the adaptation works and it is valuable to the users, it is vital to ensure a scientific process and to provide common evaluation mechanisms, methods and metrics. Taking inspiration from similar research communities and building upon current evaluation approaches, a question arises as to whether it is possible to create such a community and develop a common evaluation mechanism of useradaptive systems. In this paper, we indicate how we can move towards comparative evaluations of user-adaptive systems.